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ABSTRACT 

The paper deals with the minimax problem rain max f^(x). We work 

x 6 T& 1 i=l , • * • ,m 

with its equivalent representation min t s.t. f^(x) - t ^<_ 0 for all i. 
For this problem we design a new active set strategy in which there are three 
types of functions: active, semi-active, and non-active. This technique will 

help in preventing zigzagging which often occurs when an active set strategy 
is used* Some of the inequality constraints are handled with slack variables* 
Also a trust region strategy is used in which at each iteration there is a 
sphere around the current point in which the local approximation of the 
function is trusted. The algorithm suggested in the paper was implemented 
into a successful computer program. Numerical results are provided. 
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INTRODUCTION 


This paper deals with the minimax problem min max f-^(x) 

x e tf i=l , • • • ,m 

where f^, i=l,** # ,m are real valued functions defined on if 1 . We begin by 
transforming the problem into an equivalent inequality constrained 
minimization problem min t s.t. f ^ (x) - t < 0 for all i, i=l,* # *,ra. For 
this problem we suggest a new active set strategy in which there are three 
types of functions: nonactive, semi-active and active and these sets play a 
different role in our algorithm. The active ones are treated as equality 
constraints; the semi-active ones are assigned slack variables so that they 
can be treated as equalities too. The introduction of semi-active functions 
may help prevent the possibility of zigzagging that sometimes occurs in 
algorithms that use active set strategy. 

At the end we solve an equality constrained minimization problem for 
which we design a trust region algorithm that takes into advantage the special 
structure of the problem. In this algorithm we have at every iteration a 
sphere of radius r, in which the local model that is used to approximate the 
functions is trusted. 

Section 2 contains the basic model with all the necessary notation as 
well as the introduction of the new active set strategy. In Section 3 we give 
a description of the trust region strategy in unconstrained minimization and 
in constrained minimization. We suggest the use of the trust region for the 
minimax problem in Section 4. In Section 5 we discuss our numerical 
implementation of the algorithm and in Section 6 we give the numerical results 
of six problems taken from the literature with various starting points. 
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2. THE BASIC MODEL 

Consider m real valued functions defined on R n . We are 

I m 

interested in solving the problem 

(PI) min max f^(x) 

x e if 1 i=l , • • • ,m 

(PI) is equivalent to (P2): 

(P2) min t 

x,t 

subject to f ± (x) - t < 0, i=l,***,m. 

By introducing m slack variables we obtain another equivalent 

problem: 


(P3) 


min t 

x,t,w 

subject to f^Cx) - t + 



= 0 , 


1 - 1 . 


y m • 


We have thus transformed our original problem into a problem of equality 
constrained minimization for which successful algorithms are available. We 
use a trust region approach to solve (P3), while taking advantage of the 
structure of the problem. We demonstrate that the addition of m variables 
in (P3) does not result in additional work and provides a good way of dealing 
with the inequality constraints in (P2) . 

The following notation will be used: 
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Associated with each index j we have a Lagrange multiplier v 
we obtain the Lagrangian function 


L(x, t ,v,w) = t + l v_ L (f 1 (x) - t + V 2 wj ) . 


The gradient of L is 


VL = V L 

x,t,w,v 


VFv 

1 - 


(f i (x) - t + 1 / 2 W^)» 



and the Hessian matrix of L is 



0 

0 

0 

-e 


0 

0 


diag(v i ) 
diag(w ± ) 


VF 


-e 


diag(w ± ) 
0 


T 2 

where e =(1,*»»,1), B = Zv^ f^x). 

We now state the necessary conditions for a solution to (P2), 


Theorem 2.1. 


Let 


£ if 1 and assume that 




rank. Necessary conditions for a local minimum at x (with 

^ ^ “ft 

t = max fj(x )) are: there exists v e iP such that 


, . Thus 


is of full 
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VF (x ) 


j + u* r ° 0, 


*, * * 


v i(f±(x ) - t ) = 0 for all i, 


* 

v ± > 0 


for all i. 


f ± (x ) - t < 0 


for all i, 


* x 

and for all z e such that z = 0, zV B ^\z > 0. 


(The last condition in the theorem will always be satisfied in our algorithm 

because we use a BFGS matrix update to obtain a symmetric positive-definite 
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approximation to Ev^V f^(x). For an appropriate discussion see Powell [5].) 

At the solution (x ,v ), some components of v are zero, and the 

corresponding functions do not influence the direction of convergence. This 
leads to a natural division of the functions into active and non-active ones, 
and to an active-set strategy to determine the active set at each iteration. 

We suggest here a division of the functions into three sets: active 
functions, semi-active functions and non-active functions. These sets are 
denoted by (I), (II), and (III) respectively. A function ^ becomes active 
at a certain iteration if f (x) = max f.(x), or if it was semi-active and 

j J 

prevented us from taking a longer step in the previous iteration (i.e., 
f (x + Ax) was greater than max f (x + Ax) for II Axil > II Axil.) It will stay 

j j 

active as long as its Lagrange multiplier remains positive. Set (I) will not 
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contain more than (n+1) functions at any iteration (since at most (n+1) can be 
active at the solution) . 

A function f ± becomes semi-active if it has just been dropped from set 
(I) (in which case its multiplier is reinitialized with a positive value), or 
if it was non-active and prevented us from taking a longer step in the 
previous iteration. It will stay semi-active as long as its associated 
multiplier remains positive. In a neighborhood of the solution we expect to 
have no semi-active functions. 

We now consider the problem 

min t 
x,t,w 

(f (x) - t = 0 , i active 

|f .(x) - t +V 2 «^ = 0, i semi-active. 

Let F^, F£ denote the function vector for sets (I), (II) respectively and 
VFp VF 2 their gradients (in columns). Each active function f^ will be 

associated with Lagrange multiplier v^ and each semi-active function fj 

will be associated with multiplier Uj. Now the Lagrangian becomes 

L (x,t ,w,u,v) = t + l v (f (x) - t) + l u (f (x) - t +V 2 W^). 

i e (I) 1 1 i e (II) 11 


( 2 . 1 ) 
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r vf jv + vf 2 u 


1 - Ev. - Eu. 
1 l 


VL = 


(u i w 1 ) 


i e (II) 


( 2 . 2 ) 


( f (x) - t) , i e (I) 


L ( f ± (x) - t + V 2 ) , i e (II) J 


V 2 L = 


VF 

VF 


0 

T 


0 

0 

-e 

-e 


0 

0 

diag(u 1 ) 

0 

diag(w i ) 


VF 

-e 

0 

0 

0 


vf 2 

e T 

diag(w jL ) 

0 

0 


(2.3) 


Assume we have in a certain iteration x,t,w,v,u. A Newton-type step would 
then be determined by 


Ax 

At 

Aw 

Av 

Au 


(V 2 L) -1 VL. 


(2.4) 


When we multiply by 


V 2 L 


from the left and consider the resulting 


equation by components we obtain: 
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( B 0 

0 0 



(2.5) 


U i Aw i + U +i W i = 


for all i e (II), 


( 2 . 6 ) 



(F x " te) = 0, 


(2.7) 


where 



T 



+ diag(w^Aw^) = 0, 


v + = v + Av, 


u. 


= u + Au« 


( 2 . 8 ) 


2 

In (2,8) we assumed that for all i e (II) f^(x) - t + V2 W ^ = 0 because in 
each iteration we will take t = m4x f^(x) anc * V 2 ( w -^) = t - f^(x). 

We can now eliminate the slack variables from our system: 


from (2.6) 



w. 

1 


u 


i 


> 


from (2.8) 




2 

diag(w 1 /u 1 )u + 


0 . 


0 

w. = 2(max f.(x) 

1 j J 


f i (x)), 


Since 
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( VF 2 \T /Ax^ 
-, T / Ut I 


(2.9) 


We can now replace u + in (2.5). Define 


C = 


/B O' 


,0 0 


rv f. 


+ I T | diag(u 1 /2(max f..(x) - f ± (x)) 


'vf 2 t\ 


, -e 


( 2 . 10 ) 



( 2 . 11 ) 


In this last representation of the system (2.4) the semi-active functions 
affect only the matrix C. 

An iterative algorithm that is based on (2.4) will converge to a solution 
only if (x^,t^,w^,v^,u^) is close enough to the solution. In order to 
obtain convergence from bad starting points we suggest the use of the trust 
region strategy applied to problem (P4). 


3. THE TRUST REGION STRATEGY 

We first describe the trust region in the unconstrained case. The 

problem is min f(x) where f : R n H. For a step Ax with HAxO 
x e If 1 


small enough we have 
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f(x + Ax) « q(x + Ax) = f(x) + Vf(x)' I “Ax + V 2 Ax^BAx, (3.1) 

where Vf(x) is the gradient of f at the point x and B is a symmetric 

positive definite matrix approximating the Hessian of f at the point x. If 
we try to minimize the quadratic function q over Ax we will obtain 
Ax = -B~*Vf(x) , the quasi-Newton step. The quadratic approximation in (3.1) 
is only valid for HAxH small enough. Ax = - B ^Vf(x) may not always be a 
good step to take. In a trust region algorithm we assume that at each 
iteration we have a radius r that was determined at the end of the previous 

iteration as an estimate on the radius of the ball with a center at x in 

which the approximation (3.1) can be trusted. Thus we obtain the problem 

min q(x + Ax) = f(x) + Vf(x) T Ax +V 2 ^x^BAx 
Ax 

s.t. II Ax H < r. 

The solution to this problem is Ax = Ax (A) = -(B + A I) ^Vf(x) where 

A = 0 if II Ax ( 0 ) II < r 

A > 0 is s.t. UAx(A)II = r otherwise . 


When the radius is large enough, A = 0 and the full quasi-Newton step is 
taken. We can prove that in the neighborhood of the solution, when a BFGS 
matrix update is used to obtain B, A = 0 in each iteration resulting in 
Q-super-linear convergence. When r is very small, A » 0, Ax w j Vf (x), 
and we obtain a short step in the negative gradient direction. More details 
on this algorithm including numerical results can be found in Vardi [8]. 
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Consider next the problem 


min f(x) 

x e I? subject to h^Cx) = 0 i=l,...m 


where f,h^,...,h m : if 1 1R, m < n. When we assign a Lagrange multilplier 

V£ to each of the constraints we can form the Lagrangian function 
L(x,v) = f(x) + h(x) T v. 


VL(x,v) = 


'VL(x,v)' 


,V (X>V) ' 


^Vf (x) + Vh(x)v 
h(x) 


Let 


V L(x, v) = 


^V 2 f(x) + Zv 1 V 2 h 1 (x) 


Vh(x) 


B » V 2 f(x) + Sv i V 2 h i (x) 


be a symmetric positive definite matrix 


Vh(x) ' 


B = 


/ B 

Vh(x)\ 

\Vh(x) T 

o / 


Then the quasi-Newton step for this problem becomes 


'Ax\ 


. Av , 


= - B *VL(x,v). 
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or 

(\ 

\Vh(x) T 

The following quadratic programming problem is equivalent to (3*2): 

(Vh(x) is assumed to be of full rank; B positive definite.) 

min L(x,v) + V L(x,v)^Ax + V 2 Ax^BAx (3.3a) 

Ax X 

subject to h(x) + Vh(x)^Ax = 0. (3.3b) 

To this problem we now add the contraint OAx^ ^ r» i.e., we impose a 
trust region of radius r. Since it is possible that 

{Ax : h(x) + VhfaJ^Ax = O} D {Ax : H Ax II < r} = 

we have to make the following correction: solve at each iteration 

min L(x,v) T Ax + V 2 Ax T BAx 

X m 

s.t. ah(x) + Vh(x) Ax = 0 

II Axil < r 

where 1 > a > 0 depends on r and is determined so that 

{Ax : cch(x) + Vh(x) T Ax = O} H {Ax : 11 Ax H < r} * <t>. 
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and that the norm of the resulting step Ax is a monotonically increasing 
function of r (so that for each r there will be a unique solution Ax). 
Given r we determine X as follows 

■ 0. if HAx(\)H < r 

k > 0 such that IIAx(\)ll = r otherwise. 

The solution just presented can be written, similarly to (3.2) as 


B + XI 

Vh(x)\ 

/Ax 

Vh(x) T 

0 

\Av 


VxL 

a(\)h(x) 


(3.4) 


When the radius is large enough, X = 0 and the full quasi-Newton step is 
taken. We can prove that in the neighborhood of the solution X = 0 in each 
iteration, resulting in two-step superlinear convergence. When r is very 
small, X » 0, v + » v + = -[Vh(x)^Vh(x) ] ^Vh(x) F Vf (x) and Ax » - V x L(x,v + ). 

This algorithm was introduced in Vardi [7]. Proofs of global and local 
convergence are provided there together with numerical results for several 
test problems. 


4. THE TRUST REGION MODEL FOR THE MINIMAX PROBLEM 

We will apply the trust region strategy on the system (2.10). Let mmj 
be the number of active functions in a given iteration. We assume in this 


/ VF l\ / VF l\ 

section that I ^1 is of full rank. (The case where I 

" e / VF -| \ e ' 
deficient will be discussed in section 5.) Let Q( 1 )n 

/VF \ \-e T / 

a Q-R decomposition of I T ) where Q is an orthogc 


is rank 


represent 


orthogonal matrix, n a 


c n 

permutation matrix and T upper triangular. Let Q = [q 1 ] where has 
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mm^ rows. The new system becomes then 



where 


where 


ot(A) = min 




IIQ 2 CQ^T" T n T (F 1 - te ) II IQ 2 (J)l 
DT~ T n T (F 1 - te ) II 2 


(4.1) 


(When mm^ = n+1 we always have Q 2 (0) = see remar ^ at Theorem 4.2.) In 
(4.1), X = 0 if the radius is large enough so that the norm of 

(computed for X = 0) is less than r; otherwise X > 0 is determined so 
that H^ll = r. The next theorem summarizes the characteristics of )> 

Av(X) as a function of X. 



Theorem 4.1. Consider 


/Ax(X) 

l At(X) 


), Av(X) 


as defined in (4.1). Then 


II 


Ax(X) 

At(X) 


is monotonically decreasing to zero as a function of X. When X -*• <*>, 
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and 


/ VF l\ T 

l T ) 

\-e / 



-1 


e, 





Also when 



and A -*■ 



Proof . The proof of monotonicity of 
omitted here. For a proof see Vardi [7]. 
From (4.1) we have 


At(A) 


is simple but long and is 


(C + AI) 



(4.2) 




a(A)(F 1 - te) 


0 . 


From (4.2) we have 



(4.3) 


(4.4) 


and from (4.3) 
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(4.5) 


When X », 




From (4.2) 





Note that when 




(using the definition of Q 2 ) 



|= Q^Q 2I ,= °' 

Vl. 


thus 



= 0 . 


Further, careful analysis reveals that when Q 2 ( ) = 0 and X «• 
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Theorem 4.1 points out a nice feature of the algorithm: on the limit, 

when \ •> ®, (^) and Av depend solely on information that comes from 

active functions while semi -active functions and the matrix B are ignored. 

In the course of the algorithm we often check whether for a given r the 

resulting step, Ax satisfies max f .(x+Ax) < max f .(x). If not, we try a 

u 3 3 3 3 

shorter radius. The next theorem confirms that there always exists r > 0 

small enough (or equivalently, because ll^^ll is monotonically decreasing, 

\ > 0 large enough) such that the step is accepted. 


Theorem 4.2. Let 


C>>). MX) 


be computed as in (4.1) and assume 



* 0 . 


Also assume that if 


Q 2 (i) = 0 then (v + ) 1 > o 


for all i e I such that f^(x) < max f (x) . Then there exists \ > 0 large 


enough such that 


max f.(x + Ax(\)) < max f .(x) . 

j J j 3 
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AM 

Remark. When rank ( ^ 1 = n + 1, does not exist. However from now on we 


v-e 


will assume that the case where Q 9 (?) - 0 includes the case where 

/ VF A 2 1 

rank f T J - n + 1. 


Proof. From Taylor's theorem we have 


Xf i (x + Ax) - Xf i (x) = XVf i (x) i Ax + \Ax T V 2 f i (t])Ax for all i e (I). 


when X ®, 


XF x (x + Ax) - XF 1 (x) -» - VF^x) VF 1 (x)v + . 


if q 2 (J) * 0 


VF 


^(x) VF^(x)v + = [VF^(x) 1 VF^(x) ± ee A ] [VF^ (x) A VF^(x) + ee^] ^ e 


= ^1 - e T [VF 1 (x) T VF 1 (x) + ee T ] 1 e^ > 0. 


Thus for X large enough, for all i e I, f^(x + Ax) < f^(x). 

Q 2 (J) = 0 implies (see Theorem 4.1) that ^ ^v + + ^ ^ = 0. In this 

case we assumed that (F^ - t e) * 0 and (v + ) i > 0 for all i such that 

f.(x) < max f (x). 

j J 

In Theorem 4.1 we have shown that 


X At + -max{l,z}e^ 


7 f A t / 7 f iV ' 1 


-e 


v-e 


(F^ - te) = -max{l,z}v + (F 1 -te) < 0. 
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Thus for X large enough At < 0. 


r nr, / Vf .\ T /Ax\ ,/Ax' 

[f ± (x+Ax) - (t+At)] = [ f ^ (x) “t ] +( t) ( ) + 4f 

\-e / \At/ \At / 


Ax\ T /V 2 f 1 (n) 0 \ / Ax^ 


0/ \At> 


for all i e (II). When X -*• 


X 2 {[f i (x+Ax) - (t+At)e] - [f^x) - t e]} 
max { 1 , z } 


^i\V 7F r 


\-e7 W‘ 


7F i\7 VF r 


-e T / \-e T 


-1 


(F. - t e) < 0. 


Thus for X large enough 


Fj(x + Ax) < Fj(x) + Ate < F J (x) 


This completes the proof. 


We can now present the algorithm: 


Step 1 


Start with x°, (II) = <J> , (I 
k = -1. 


{i: ^(x 0 ) = max f^x 0 )}, r°, B°, 
J 


Step 2 Let k = k+1, RADINC = 0. 


Step 3 


/VF (x k )\ 

Take a Q-R decomposition of ( T ) and check whether 
q, -e I 

Q2 (l ) = 0# If so > compute v + . If for all i e I such that f ± (x) 

< max f j (x) (v + ) ± > 0 (or if Q 2 (°) * 0) continue to Step 4. 
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Otherwise remove from set I the function ig such that f^ (x) < 
max fj(x) and with the minimal (v + ) ± , and insert it in set II. 
Restart step 3. 


Step 4 Compute as in (2* 10)* 


Step 5 


Find 


/A kv 

/Ax \ 

> I v I 9 
\At k / 


requirements: 

Section 5 for computational details*) 


Av that solve the system (4*1) with the 


Ax k , , , / Ax k 

, II < r k , X k > 0, A k (ll I - r"l = 0. 
At k \ At 


(See 


Step 6 Check if max f (x k + Ax k ) < max f (x k ). If so continue to Step 

j J 3 J 

7. If not, if RAD INC < 0 half r k , RAD INC = RAD INC- 1 and go to 
Step 5. (If RADINC > 0 the step corresponding to the smaller 
radius has been computed already* Retrieve it and continue to Step 
7.) 


Step 7 If no new active functions are introduced by the last step and if 
RADINC > 0 and if X k > 0 let RADINC = RADINC + 1, store the 

current step, double r k and go to Step 4. Otherwise continue to 
step 8. 


Step 8 Check for convergence. 

Step 9 Compute u k+1 by (2.9) and v k+1 by (4.5) and according to the 

k+1 

signs update (I) and (II). (See Section 2). Compute B 
(details in section 5) . 
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Step 10 x* 


lc Ic 

x + Ax . Return to Step 2. 


Theorem 4.2 guarantees that in every iteration there will be r k small 

enough for which max f (x k+1 ) < max f .(x k ). We have to justify here the 

3 3 3 3 

removal of an active constraint as described in Step 3 when (v + > 1 < 0 and 
^2^1 ) = ® (i.e. nraij = n+1. For a discussion on the possiblility of 

0 / VF i\ 

^2^1^ = ® when ( ^ ) is rank deficient see Section 5.) 

The necessary conditions at the solution x require that 


Since mm j = n+1 


0\ /VF^x*) 


v* = 0 and v* > 0, 


j* = - 


V F i ( x * ) \" 1 /0 


(because F^x*) = t*). Thus Q 2 (J) = 0 and (v + ) < 0 implies that we are 

converging to a point with a negative multiplier and we therefore need to 
remove that function from the active list. 


5. IMPLEMENTATION 

A scali ng problem In order to prevent a scaling problem that may be 

/ 7F i\ 

created in I T j when H VF ^ 0 is much greater or much less than one, it is 
important to actually replace (P2) by 


(P2') 


min c t 

subject to f^x) - c t < 0 


i = 1, • • • ,m 
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Now instead of working with the matrix ^ ^ j we work with the matrix 

In order for this matrix to be well scaled, we update c to 
c = llVF^(x)ll frequently. This modification can be accomplished without 
difficulties. (We will continue to assume c = 1 in the rest of the paper 
for clarity.) 



Initialization of (I) and (II). As 'Step 1' in the algorithm indicates, at 
the first iteration usually only one function is active and all the rest are 
nonactive. We have tried other options where more of the functions were 
active and semi-active in the first iterations but observed that this strategy 
does not result in a decrease in the number of iterations. Since these other 
options required in total more gradient evaluations we decided to use 
initially one active function. 


Solving System (4.1). 


Using the Q-R decomposition of the matrix 
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Q 




we can obtain the following expressions: 


From (4.3) 


Qi ( a ) - -a( A)T -T JI T (Fj - t e) . 


From (4.2) 


/Ax 


\At 


) = 


2 CQ2 + A I) _1 Q 2 



(5.1) 


(5.2) 



A Cholesky decomposition of C is performed, C = LL T . (When mm, = 0 


and C = 

T v _ r R 


-B 0- 

, L = 

" L 1 

0- 

-0 0. 


-0 

0. 

J represent 

a 

Q-R 


where B - L^L^.) Define M = C^L and let 


permutation, R upper triangular). Partition P into P = 

■\T I T\ r/nT, 


L P 2 


where 


has n+l-mmj rows. We now have (Q^Q^ + x I) = 2(R i R + X I)E A and Q 2 (^) 
can be obtained by solving 


R 


X V 2l 


T r 


R 

>1 


N 


Ax\ 

At/ 


R 

v 2 


where 


J T \ nr» rn l / Ax ^ 

b - R VQoI 1 + PjL A qJ Iq 1 

\At , 


In order to solve this linear least squares problem, a Q-R decomposition of 
R 

is obtained with the use of Givens transformation. 


* /2 Ij Ax 

Define < K^) = - r. For a given X we can use the above 

decomposition to compute <KX) and <j>'(X). A X iterative scheme is used to 

obtain X such that <KX) =0. In this X-iterative process we usually have 

a good guess for X and we take X J+1 = X^ - r — ^ ^ . Known upper 

r <T(X J ) 

and lower bounds on the solution are used to help the convergence. It takes 


an average of less than two X-iterations to obtain an acceptable solution, 
(i.e. X such that <J>(A) < 0.1 * r) . For more details on this part of the 
algorithm see Vardi [7] . 
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Updating B. When moving from the k tk iteration to the (k+l) tk iteration, 


B k - y v) v 2 f (x k ) + y u k v 2 f . 

^ , i xx i Lu , i xx i 


(x ) 


i e I 


i e II 


has to be updated into 


„k+l 


V vf 1 V 2 £ r„ k ) + V u k+1 V 2 fix k+1 ). 

Z— / . i xx i 1 - J L-u . , . i xx i 1 - 7 

i c I k i e II k+1 


Define 


y k = [VF 1 (x k+1 ) - VF 1 (x k )]v k+1 + [VF 2 (x k+i ) - VF,,(x k )]u 


k, i k+1 


k+I, 


kv l k+1 


/ Ax \ k+1 k k 

If I - I I s small, it is reasonable to require that B Ax = y . Other 

\At k / 

k+1 

requirements on B are that it is symmetric, positive definite, close to 

k 

B in some norm and easy to compute. The B.F.G.S. update is often used in 
which 


i , k kT B k A k A kT _k 

B t+l ’ BFGS ( B k , Ax k »u k } - B k + V7T- B .% 5 7 £ 

y Ax Ax B Ax 


1 _ VT tr k+1 

If B^ is positive definite and y Ax > 0, B is positive definite. In 

our algorithm we use a modification of the BFGS that was suggested by Powell 

(see Powell [5]): 

B‘ 

where 


■ k+l - BFGS(B k ,Ax k ,y k ) 


y k - 9y k + (l-e>B k Ax k 


and 
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n Q . kT _k . k 
0.8 Ax B Ax 


A kT _k A k 
Ax B Ax 


kT A k 
y Ax 


k k . n ,, *kT _k . k 
if y Ax > 0.2 A BAx 


otherwise 


Powell proved that when this update is used in an algorithm for constrained 
minimization, local two-step superlinear convergence results. 


VFi 

When ( T ) is rank deficient 


VF. T 

When the Q-R decomposition is performed to obtain Q( ™)n = ( ), we 

-e 0 

check at each step whether any of the remaining columns have norm larger than 

(machine ps) x If not, we stop. If JL steps were performed, and T is 

VF-i . 

an JL x mm., upper triangular matrix, we consider rank ( i) = A. Partition 
1 -e 1 

Q = (q^ ) > n = n x ,n 2 and T = [T,S] where has JL rows, 11^ and T 

have JL columns. (5.1) becomes Q 1 (jj) = - a(X)T~ T n 2 (F 1 - t e) and all 

other expressions remain the same. The effect of this is that the active 

VFi 

functions that correspond to the remaining columns in ( i) are ignored. 

VFl ” e 
( •j>) rank deficient it is possible that Q ? (i) = 0 and as the 

-e i 1 

algorithm indicates v^ is checked and active functions with negative (v^_) 

may become semiactive. 


6. NUMERICAL TESTS 

A computer program based on the algorithm was written and tested on 6 
test problems from the optimization literature as well as one large practical 
problem. 

The complete information on these problems is given in this section. In 
order to check global convergence we added starting points that are much 
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further from the known solution than the suggested starting points. In all 
problems global convergence was achieved and fast local convergence was 
observed. 

For all problems we used r® = 1, B® = I. The stopping criteria were 


1 . 


IIAxll < 10 10 ( llxll + 1), 


2 . 




< 10 “ 6 , 


and 


max f (x) - min f . (x) < 10 ^ max{ | max f (x) I , | min f (x) | } . 

iel 1 i e I 1 iel 1 iel 1 


In each of the problems we tried 3 starting points. We recorded the 
number of function evaluations and the number of individual gradient 
evaluations (i.e., gradients of functions in (I) and (II); if, for example, 
thre are in a certain iterations mm^ active functions and mm 2 semi-active 
functions, we count mm^ + mm 2 gradient evaluations.) 

We display the count for all iterations at the end of which a change in 


(I) or (II) occurred. 
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Test Problem 1 


n - 2, m = 3 F^x) = x 2 + x* 

F 2 (x) = (2 - x L ) 2 + (2 - x 2 ) 2 
F 3 (x) = 2 exp(-x^ + x 2 ) 


solution x = 


f 


1 “ 


(1.139037652, .8995599384) 
F 2 = 1.952224494 


Starting 

Point 

Iteration 

No* 

No of* Fun. 
Evaluation 

No. of 
Ind. Grad. 
Evaluation 

max fj( x ) 

Semi- 

Active 

Functions 

Active 

Functions 

d,-.l) 

0 

1 

1 

5.41 

- 

2 


2 

6 

3 

2.170849382 

- 

2 


7 

12 

13 

1.952224494 

- 

1,2 

(10,-1) 

0 

1 

1 

101 

- 

1 


1 

6 

2 

2.335229368 

- 

1 


7 

12 

14 

1.952224494 

- 

1,2 

(100,-10) 

0 

1 

1 

20000. 

- 

1 


3 

14 

4 

40.68514215 

- 

1 


5 

17 

8 

2.501544410 

3 

1 


7 

19 

12 

2.010184030 

- 

1,3 


8 

20 

15 

1.964914927 

3 

1,2 


12 

24 

23 

1.952224494 

- 

1,2 
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Test Problem 2 


n = 2, m = 3 


4,2 

x^ + X2 

as in example 1 



solution x = (1,1) 


Starting Iteration 
Point No. 

No. of Fun. 
Evaluation 

No. of 
Ind . Grad . 
Evaluation 

Semi 

max fj( x ) Active 

J Functions 

Active 

Functions 

(l,-.l) 0 

1 

1 

5.41 

- 

2 

1 

2 

2 

3.181388755 

- 

2 

2 

4 

4 

2.417247765 


1,2 

4 

6 

10 

2.004795643 

3 

1,2 

6 

9 

16 

2.000000000 

- 

1,2,3 

(10,-1) 0 

1 

1 

10001. 

- 

1 

1 

6 

2 

19.74943380 

- 

1 

2 

8 

4 

19.36287258 

- 

1,2 

6 

14 

16 

2.808076541 

1 

2,3 

10 

18 

28 

2.000000000 

- 

1,2,3 

(100,-10) 0 

1 

1 

100000100. 

- 

1 

10 

20 

11 

4.436401306 


1 

13 

26 

17 

2.076370586 

- 

1,2 

15 

31 

23 

2.000020450 

3 

1,2 

17 

33 

29 

2.000000000 

- 

1,2,3 
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n = 4, m = 4 


Test Problem 3 


F^x) = x* + x* + 2x^ + x* - 5x x - 5 x 2 - 21x 3 + 7 X/ , 

F 2 (x) - F : (x) - 10(-xJ - x 2 ~ *3 - x* - x x + x 2 - x 3 + x 4 + 8) 

F 3 (x) = F^x) - 10(-xJ - 2 x 2 - x 3 - 2x 2 A + x i + x 4 + 10 ) 

F 4 (x) = F^x) - 10(-2xJ - x\-x\- 2xj_ + x 2 + x 4 + 5) 

solution x = (0,1,2, -1) F x = f 2 = F 4 = -44. 


Starting Iteration 
Point No. 

No. of Fun. 
Evaluation 

No. of 
Ind. Grad. 
Evaluation 

Semi 

max f (x) Active 

Functions 

Active 

Functions 

(0,0, 0,0) 0 

1 

1 

0 

- 

1 

2 

6 

3 

-39.35982740 

- 

1 

3 

8 

5 

-41.13057951 


1,4 

5 

11 

11 

-41.67511609 

2 

1,4 

10 

16 

26 

-44.00000000 

- 

1 , 2,4 

(10,10,10,10) 0 

1 

1 

5960 

- 

3 

1 

6 

2 

973.8877654 


3 

2 

8 

4 

91.01083219 


2,3 

4 

12 

8 

-13.00041510 

2 

3 

7 

20 

20 

-41.13646654 

1,2,3 

4 

9 

24 

26 

-43.41889604 

2 

1,4 

15 

31 

44 

-44.00000000 

- 

1 , 2,4 

(100,100,100,100) 






0 

1 

1 

645500. 

- 

3 

4 

18 

5 

344.6890778 

- 

3 

8 

23 

13 

-22.66648305 

- 

3,4 

9 

24 

16 

-27.19599155 

- 

1 , 3,4 

10 

25 

20 

-39.22987562 

3 

1 , 2,4 

18 

34 

44 

-44.00000000 

- 

1 , 2,4 
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Test Problem 4 


F x (x) 

F 2 (x) 


2 . 2 

X 1 + x 2 + x l x 2 
sin(x^) 

cos(x 2 ) 


Solution x = ±(.4532962370, -.9065924741) 
F x = f 3 = .6164324356 
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Test Problem 5 


n = 3, m = 6 

F 1 (x) = x 2 + x 2 + x 2 - 1 

F^(x) = x i + x 2 “ x 3 + 1 


F 2 (x) = x 2 + x 2 + (x 3 ~2) 2 

F^(x) = 2x^ + 6x 2 + 2(5x 3 ~Xj+l) 2 


F 3 (x) = x i + x 2 + x 3 ~ 1 

F 6 (x) = x 2 - 9x 3 


Solution x = (.32825995, 0, .1313200636) 
F 2 = F 5 = 3.599719300 


Starting 

Point 

Iteration 

No. 

No. of Fun. 
Evaluation 

No. of 

Ind. Grad, max f (x) 
Evaluation 

Semi 

Active 

Functions 

Active 

Functions 

(1,1,1) 

0 

i 

i 

58. 

- 

5 


2 

4 

3 

5.028346958 

- 

5 


13 

26 

25 

3.599719300 

- 

2,5 

(10,10,10) 0 

1 

1 

5962. 

- 

5 


5 

14 

6 

4.448679576 

- 

5 


18 

43 

32 

3.599719300 

- 

2,5 

(100,100, 

100) 0 

1 

1 

2381602. 

- 

5 


1 

10 

2 

21172.51683 

- 

5 


2 

12 

4 

13296.67178 

- 

1,5 


3 

13 

6 

7506.056041 

5 

1 


4 

14 

7 

1785.905159 

- 

1 


5 

15 

9 

848.2530772 

- 

1,2 


6 

16 

11 

324.5466404 

1 

2 


7 

17 

12 

213.7819513 

- 

2 


15 

25 

28 

3.599719300 

- 

2,5 


Test Problem 6 


n = 3, m = 30 


u i 

F.(x) = -y . + x , + — 

i 1 VjX 2 + w^x 3 


j = 1» * * * > 15 


F j (x) = -F (j-15) (x) 


j = 16, • • • , 30 


where 


u. = 1, v = 16 - i and w. = min{u.,v.}, 

J J j J J 3 3 

y = (.14, .18, .22, .25, .32, .39, .37, .58, .73, .96, 1.34, 2.1, 4. 39) 


Solutions : 


x = ±(.05346938776, t, 3.5 - t) .5 < t < 1.5 

F 9 = F 23 = F 30 = .05081632653 (or F 24 = F g = F 15 ) 


x = ±(.2831587485, -4.8412419079, 9.323361111) 

F 6 ' F 20 ' F 30 ' - 7602 099910 <or F 21 ‘ F 5 ' F 15> 


F 


x = ±(.02033344564, 
1 = F 2 = F 23 " F 30 


100795522614, 


3.3992044739) 

(or F 16 = F 17 = F 8 = 


08395226864 
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Test Problem 6 (continued) 


Starting Iteration 
Point No. 

No. of Fun. 
Evaluation 

No. of 

Ind. Grad, max f (x) 
Evaluation 

Semi 

Active 

Functions 

Active 

Functions 

(1,1,1) o 

i 

i 

4.11 

- 

15 

1 

3 

2 

0.7422952056 

- 

15 

2 

4 

4 

0.3286634371 

- 

9,15 

3 

5 

7 

0.2339086843 

15 

9,18 

5 

7 

15 

0.05443554245 

15 

9,18,30 

7 

11 

21 

0.05115868122 


9,18,30 

8 

12 

25 

0.05081633724 

- 

9,18,23,30 

9 

13 

29 

0.05081632653 

18 

9,23,30 

(10,10,10) 0 

1 

1 

9.86625 


1 

3 

8 

4 

2.417983420 


1 

9 

21 

16 

1.207116039 

- 

1,30 

10 

23 

19 

0.8923142963 

15 

1,30 

11 

24 

22 

0.1098360412 

- 

1,15,30 

12 

27 

26 

0.08548202468 

- 

1,9,15,30 

13 

29 

30 

0.07228591795 

1,15 

9,30 

16 

34 

39 

0.05081632653 

- 

9,23,30 

(100,100,100) 






0 

1 

1 

99.860625 

- 

1 

5 

13 

6 

11.35666692 

- 

1 

16 

42 

28 

1.779837231 

- 

1,30 

17 

43 

31 

0.6808764481 

- 

1,9,30 

18 

44 

34 

0.3756488583 

1 

9,30 

19 

47 

36 

0.1292719191 

- 

9,30 

20 

49 

39 

0.07580649840 

15 

9,30 

23 

55 

48 

0.05392409950 

- 

9,18,30 

24 

56 

52 

0.05081719777 

- 

9,18,23,30 

25 

57 

56 

0.05081632653 

18 

9,23,30 
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The algorithm has also undergone limited testing on the problem of 
designing an aircraft lateral stability augmentation system found in [9] 
identified as the deterministic problem at Mach 2.5. By using different 
starting points it both found the minimum reported in Schy et al. [1981] and 
located another local minimum. Execution times correspond favorably with 
those of the algorithm used in Schy et al. [1981]. In the only case where 
problems formulations (starting points, convergence criteria, etc.) were 
similar enough to make comparisons valid, the present algorithm executed in 
less than V 4 of the time (Giesy [1982]). In these testings convergence to a 
local minimum and fast local convergence were always achieved. 



34 


REFERENCES 


[1] M. AVRIEL. Nonlinear Programming: Analysis and Methods. Prentice-Hall, 

Inc. , 1976. 

[2] C. CHARALAMBOUS and A. R. CONN. An official method to solve the minimax 
problem directly. SIAM J, Numer. Anal., 15, No. 1 (1978), pp. 162-187. 

[3] S. P. HAN. A globally convergent method for nonlinear program. J. 
Optim. Theory Appl. , 22, No. 3 (1977), pp. 297-309. 

[4] W. MURRAY and M. L. OVERTON. A projected Lagrangian algorithm for 
nonlinear minimax optimization. System Optimization Laboratory Technical 
Report No. 79-21, 1979. 

[5] M. J. D. POWELL. The convergence of variable metric methods for 
nonlinearly constrained optimization calculations; NLP3; Meyer and 
Robinson, eds.. Academic Press, 1978. 

[6] R. A. TAPIA. On the role of slack variables in quasi— Newton methods for 
constrained optimization. Numerical Optimization of Dynamic Systems, L. 
C. W. Dixon and G. P. Szego, eds., 1980, pp. 235-246. 

[7] A. VARDI. A trust region algorithm for equality constrained 

minimization: convergence properties and implementation. ICASE Report 

No. 81-9, February 1981, to appear in SIAM J. Numer. Anal., 1985. 

[8] A. VARDI. A trust region algorithm for unconstrained minimization: 
convergence properties and implementation. ICASE Report No. 80-35, 1980. 


35 


[9] A. A. SCHY and D. P. GIESY, Multiobjective insensitive design of airplane 
control systems with uncertain parameters, AIAA Guidance and Control 
Conference, Albuquerque, New Mexico, August 19-21, 1981. 





1. Report No. NASA CR— 172382 2. Government Accession No. 

ICASE Report No. 84-25 


4. Title and Subtitle 

A New Minmax Algorithm 


7. Author(s) 

Avi Vardi 


9. Performing Organization Name and Address 

Institute for Computer Applications in Science 
and Engineering 

Mail Stop 132C, NASA Langley Research Center 
Hampton. VA 23665 


12. Sponsoring Agency Name and Address 

National Aeronautics and Space Administration 
Washington, D.C. 20546 

15. Supplementary Notes 

Langley Technical Monitor: Robert H. Tolson 

Final Report 

16. Abstract 


3. Recipient's Catalog No. 


5. Report Date 
June 1984 


6. Performing Organization Code 


8. Performing Organization Report No. 

84-25 


10. Work Unit No. 


11. Contract or Grant No. 

NAS1-15810 


13. Type of Report and Period Covered 
Contractor Report 

14. Sponsoring Agency Code 

505-31-83-01 


The paper deals with the minimax problem min max f^(x). 

x e 1? i=l , • • • ,m 

We work with its equivalent representation min t s.t. f j _(x) - t < 0 for all i. 
For this problem we design a new active set strategy in which there "are three types 
of functions. active, semi-active, and non— active. This technique will help in 

preventing zigzagging which often occurs when an active set strategy is used. Some 
of the inequality constraints are handled with slack variables. Also a trust region 
strategy is used in which at each iteration there is a sphere around the current 
point in which the local approximation of the function is trusted. The algorithm 
suggested in the paper was implemented into a successful computer program. Numerical 
results are provided. 


17. Key Words (Suggested by Author(s)) 

minimization constraints 


18. Distribution Statement 


64 Numerical Analysis 
61 Computer Programming & Software 

Unclassified - Unlimited 


19. Security Classif. (of this report) 

20. Security Classif. (of this page) 

21. No. of Pages 

22. Price 

Unclassified 

Unclassified 

37 

A0 3 


For sale by the National Technical Information Service, Springfield, Virginia 22161 


NASA-Langley, 1984 













